<!doctype html>
<html lang="zh-CN">
<head>

    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    
    <meta name="referrer" content="no-referrer-when-downgrade">
    

    <title>Calico网络之BGP详解 | Sang的博客</title>
    <meta property="og:title" content="Calico网络之BGP详解 - Sang的博客">
    <meta property="og:type" content="article">
        
    <meta property="article:published_time" content='2022-02-08T11:32:18&#43;08:00'>
        
        
    <meta property="article:modified_time" content='2022-02-08T11:32:18&#43;08:00'>
        
    <meta name="Keywords" content="golang,go语言,shell,k8s,博客,python,软件架构,公众号">
    <meta name="description" content="Calico网络之BGP详解">
        
    <meta name="author" content="仨哥">
    <meta property="og:url" content="https://stto_32.gitee.io/site/post/calico/">
    <link rel="shortcut icon" href='/site/favicon.ico'  type="image/x-icon">

    <link rel="stylesheet" href='/site/css/normalize.css'>
    <link rel="stylesheet" href='/site/css/style.css'>
    <script type="text/javascript" src="//cdn.bootcdn.net/ajax/libs/jquery/3.4.1/jquery.min.js"></script>

    
    
    
        <link href="https://cdn.bootcdn.net/ajax/libs/fancybox/3.5.7/jquery.fancybox.min.css" rel="stylesheet">
    
    
    
    
        <link rel="stylesheet" href='/site/css/douban.css'>
    
        <link rel="stylesheet" href='/site/css/other.css'>
    
</head>


<body>
    <header id="header" class="clearfix">
    <div class="container">
        <div class="col-group">
            <div class="site-name ">
                
                    <a id="logo" href="https://stto_32.gitee.io/site">
                        Sang的博客
                    </a>
                
                <p class="description">专注于Python、Go语言(golang)、云原生</p>
            </div>
            <div>
                <nav id="nav-menu" class="clearfix">
                    <a class="current" href="https://stto_32.gitee.io/site">首页</a>
                    
                    <a  href="https://stto_32.gitee.io/site/tools/" title="小工具">小工具</a>
                    
                    <a  href="https://stto_32.gitee.io/site/archives/" title="归档">归档</a>
                    
                    <a  href="https://stto_32.gitee.io/site/about/" title="关于">关于</a>
                    
                </nav>
            </div>
        </div>
    </div>
</header>

    <div id="body">
        <div class="container">
            <div class="col-group">

                <div class="col-8" id="main">
                    
<div class="res-cons">
    
    <article class="post">
        <header>
            <h1 class="post-title">Calico网络之BGP详解</h1>
        </header>
        <date class="post-meta meta-date">
            2022年2月8日
        </date>
        
        
        <div class="post-meta">
            <span id="busuanzi_container_page_pv">|<span id="busuanzi_value_page_pv"></span><span>
                    阅读</span></span>
        </div>
        
        
        <div class="post-content">
            <p>Calico是一个<code>纯三层</code>的协议，为OpenStack虚机和Docker容器提供多主机间通信。Calico不使用重叠网络比如flannel和libnetwork重叠网络驱动，使用虚拟路由代替虚拟交换，每一台虚拟路由通过BGP协议传播可达信息（路由）到剩余数据中心。</p>
<h3 id="calico架构">Calico架构</h3>
<p>架构图：
<a href="https://img2018.cnblogs.com/blog/1060878/201904/1060878-20190413152300545-538840176.png">
        <a data-fancybox="gallery" href="https://img2018.cnblogs.com/blog/1060878/201904/1060878-20190413152300545-538840176.png">
            <img class="mx-auto" alt="" src="https://img2018.cnblogs.com/blog/1060878/201904/1060878-20190413152300545-538840176.png" />
        </a>
    </a></p>
<p>Calico网络模型主要工作组件：</p>
<ul>
<li><code>Felix</code>：calico的核心组件，运行在每个节点上。主要的功能有接口管理、路由规则、ACL规则和状态报告,Felix会监听ECTD中心的存储，从它获取事件，比如说用户在这台机器上加了一个IP，或者是创建了一个容器等。用户创建pod后，Felix负责将其网卡、IP、MAC都设置好，然后在内核的路由表里面写一条，注明这个IP应该到这张网卡。同样如果用户制定了隔离策略，Felix同样会将该策略创建到ACL中，以实现隔离</li>
<li><code>etcd</code>：分布式键值存储，主要负责网络元数据一致性，确保Calico网络状态的准确性，可以与kubernetes共用；</li>
<li><code>BGP Client（BIRD）</code>：Calico 为每一台 Host 部署一个 BGP Client，它的作用是将Felix的路由信息读入内核，并通过BGP协议在集群中分发。当Felix将路由插入到Linux内核FIB中时，BGP客户端将获取这些路由并将它们分发到部署中的其他节点。这可以确保在部署时有效地路由流量</li>
<li><code>BGP Router Reflector</code>：大型网络仅仅使用 BGP client 形成 mesh 全网互联的方案就会导致规模限制，所有节点需要 N^2 个连接，为了解决这个规模问题，可以采用 BGP 的 Router Reflector 的方法，使所有 BGP Client 仅与特定 RR 节点互联并做路由同步，从而大大减少连接数</li>
<li><code>Calicoctl</code>：calico 命令行管理工具</li>
</ul>
<p><strong>架构特点</strong></p>
<p>由于Calico是一种<code>纯三层</code>的实现，因此可以避免与二层方案相关的数据包封装的操作，中间没有任何的NAT，没有任何的overlay，所以它的转发效率可能是所有方案中最高的，因为它的包直接走原生TCP/IP的协议栈，它的隔离也因为这个栈而变得好做。因为TCP/IP的协议栈提供了一整套的防火墙的规则，所以它可以通过IPTABLES的规则达到比较复杂的隔离逻辑。</p>
<h3 id="calico-工作原理">Calico 工作原理</h3>
<p>Calico把每个操作系统的协议栈认为是一个路由器，然后把所有的容器认为是连在这个路由器上的网络终端，在路由器之间跑标准的路由协议——BGP的协议，然后让它们自己去学习这个网络拓扑该如何转发。所以Calico方案其实是一个<code>纯三层</code>的方案，也就是说让每台机器的协议栈的三层去确保两个容器，跨主机容器之间的三层连通性。</p>
<p>对于控制平面，它每个节点上会运行两个主要的程序，一个是Felix，它会监听ECTD中心的存储，从它获取事件，比如说用户在这台机器上加了一个IP，或者是分配了一个容器等。接着会在这台机器上创建出一个容器，并将其网卡、IP、MAC都设置好，然后在内核的路由表里面写一条，注明这个IP应该到这张网卡。绿色部分是一个标准的路由程序，它会从内核里面获取哪一些IP的路由发生了变化，然后通过标准BGP的路由协议扩散到整个其他的宿主机上，让外界都知道这个IP在这里，你们路由的时候得到这里来。</p>
<p>由于Calico是一种<code>纯三层</code>的实现，因此可以避免与二层方案相关的数据包封装的操作，中间没有任何的NAT，没有任何的overlay，所以它的转发效率可能是所有方案中最高的，因为它的包直接走原生TCP/IP的协议栈，它的隔离也因为这个栈而变得好做。因为TCP/IP的协议栈提供了一整套的防火墙的规则，所以它可以通过IPTABLES的规则达到比较复杂的隔离逻辑。</p>
<p>Calico是纯三层的SDN 实现，它基于BPG 协议和Linux自身的路由转发机制，不依赖特殊硬件，容器通信也不依赖iptables NAT或Tunnel 等技术</p>
<h3 id="calico的两种网络模式">Calico的两种网络模式</h3>
<p><strong>IPIP</strong></p>
<p>从字面来理解，就是把一个IP数据包又套在一个IP包里，即把 IP 层封装到 IP 层的一个 tunnel。它的作用其实基本上就相当于一个基于IP层的网桥！一般来说，普通的网桥是基于mac层的，根本不需 IP，而这个 ipip 则是通过两端的路由做一个 tunnel，把两个本来不通的网络通过点对点连接起来。</p>
<p><strong>BGP</strong></p>
<p>边界网关协议（Border Gateway Protocol, BGP）是互联网上一个核心的去中心化自治路由协议。它通过维护IP路由表或‘前缀’表来实现自治系统（AS）之间的可达性，属于矢量路由协议。BGP不使用传统的内部网关协议（IGP）的指标，而使用基于路径、网络策略或规则集来决定路由。因此，它更适合被称为矢量性协议，而不是路由协议。BGP，通俗的讲就是讲接入到机房的多条线路（如电信、联通、移动等）融合为一体，实现多线单IP，BGP 机房的优点：服务器只需要设置一个IP地址，最佳访问路由是由网络上的骨干路由器根据路由跳数与其它技术指标来确定的，不会占用服务器的任何系统。</p>
<h3 id="bgp-工作模式">BGP 工作模式 </h3>
<hr>
<h4 id="bgp是怎么工作的">BGP是怎么工作的？</h4>
<p>Calico 项目提供的 BGP 网络解决方案，与 Flannel 的 host-gw 模式几乎一样。也就是说，Calico也是基于路由表实现容器数据包转发，但不同于Flannel使用flanneld进程来维护路由信息的做法，而Calico项目使用BGP协议来自动维护整个集群的路由信息</p>
<p>
        <a data-fancybox="gallery" href="https://gitee.com/stto_32/img/raw/master/20220105173708.png">
            <img class="mx-auto" alt="" src="https://gitee.com/stto_32/img/raw/master/20220105173708.png" />
        </a>
    </p>
<h4 id="修改配置">修改配置</h4>
<p>在安装calico网络时，默认安装是IPIP网络。calico.yaml文件中，将CALICO_IPV4POOL_IPIP的值修改成 &ldquo;off&rdquo;，就能够替换成BGP网络。</p>
<p>
        <a data-fancybox="gallery" href="https://gitee.com/stto_32/img/raw/master/20220105171531.png">
            <img class="mx-auto" alt="image-20220105171531663" src="https://gitee.com/stto_32/img/raw/master/20220105171531.png" />
        </a>
    </p>
<h4 id="和ipip模式对比">和IPIP模式对比</h4>
<p>BGP网络相比较IPIP网络，最大的不同之处就是<code>没有</code>隧道设备<code>tunl0</code>，pod之间的流量直接从·<code>宿主机</code>通过arp下一跳到目的地<code>宿主机</code>，减少了tunl0环节。</p>
<p>master节点上路由信息，从路由信息来看，没有tunl0设备。
 
        <a data-fancybox="gallery" href="https://gitee.com/stto_32/img/raw/master/20220106142858.png">
            <img class="mx-auto" alt="image-20220106142858029" src="https://gitee.com/stto_32/img/raw/master/20220106142858.png" />
        </a>
    
node上路由信息

        <a data-fancybox="gallery" href="https://gitee.com/stto_32/img/raw/master/20220106142843.png">
            <img class="mx-auto" alt="image-20220106142843062" src="https://gitee.com/stto_32/img/raw/master/20220106142843.png" />
        </a>
    </p>
<h4 id="53-网络抓包测试">5.3 网络抓包测试</h4>
<p>以其中2个pod为例展示,从<code>A(10.244.123.135)</code>【node01】ping<code>B(10.244.140.64)</code>【node02】</p>
<p>
        <a data-fancybox="gallery" href="https://gitee.com/stto_32/img/raw/master/20220107101821.png">
            <img class="mx-auto" alt="image-20220107101821262" src="https://gitee.com/stto_32/img/raw/master/20220107101821.png" />
        </a>
    </p>
<p>进入pod地址为 <code>10.244.123.135</code>
<code>kubectl exec -it myapp-deployment-79f56db86b-84pgs   sh -n test123123</code></p>
<p>查看路由及ip addr</p>
<div class="highlight"><div style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4">
<table style="border-spacing:0;padding:0;margin:0;border:0;width:auto;overflow:auto;display:block;"><tr><td style="vertical-align:top;padding:0;margin:0;border:0;">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f"> 1
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f"> 2
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f"> 3
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f"> 4
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f"> 5
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f"> 6
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f"> 7
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f"> 8
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f"> 9
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">10
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">11
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">12
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">13
</span></code></pre></td>
<td style="vertical-align:top;padding:0;margin:0;border:0;;width:100%">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell">/ <span style="color:#75715e"># ip route</span>
default via 169.254.1.1 dev eth0 
169.254.1.1 dev eth0 scope link 
/ <span style="color:#75715e"># ip addr</span>
1: lo: &lt;LOOPBACK,UP,LOWER_UP&gt; mtu <span style="color:#ae81ff">65536</span> qdisc noqueue state UNKNOWN qlen <span style="color:#ae81ff">1000</span>
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if34: &lt;BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN&gt; mtu <span style="color:#ae81ff">1500</span> qdisc noqueue state UP 
    link/ether 22:e9:77:e0:ed:f0 brd ff:ff:ff:ff:ff:ff
    inet 10.244.123.135/32 scope global eth0
       valid_lft forever preferred_lft forever
/ <span style="color:#75715e"># </span>
</code></pre></td></tr></table>
</div>
</div><p>我们看到pod的eth0网卡的IP地址为<code>10.244.123.135</code>,veth pair的设备为if34，又因为该pod部署在node01，因此我们在node01主机通过匹配路由<code>route -n|grep '10.244.123.135'</code>找到的设备为<code>cali52074d225a7</code>,因此我们对<code>cali52074d225a7</code>设备抓包</p>
<p>在<code>该pod所在node节点</code>上查看路由信息，重点关注这个网卡<code>cali52074d225a7</code>，<code>Flex默认</code>会为分配到该node上的pod容器生成一条<code>pod地址 calixxxx</code>的路由信息</p>
<p>
        <a data-fancybox="gallery" href="https://gitee.com/stto_32/img/raw/master/20220106143951.png">
            <img class="mx-auto" alt="image-20220106143951841" src="https://gitee.com/stto_32/img/raw/master/20220106143951.png" />
        </a>
    </p>
<p>我们在该node01上抓包<code>tcpdump -i cali52074d225a7</code></p>
<div class="highlight"><div style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4">
<table style="border-spacing:0;padding:0;margin:0;border:0;width:auto;overflow:auto;display:block;"><tr><td style="vertical-align:top;padding:0;margin:0;border:0;">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">1
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">2
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">3
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">4
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">5
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">6
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">7
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">8
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">9
</span></code></pre></td>
<td style="vertical-align:top;padding:0;margin:0;border:0;;width:100%">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-fallback" data-lang="fallback"># ping 10.244.140.64
PING 10.244.140.64 (10.244.140.64): 56 data bytes
64 bytes from 10.244.140.64: seq=0 ttl=62 time=1.184 ms
64 bytes from 10.244.140.64: seq=1 ttl=62 time=0.552 ms
^C
--- 10.244.140.64 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.552/0.868/1.184 ms
/ # 
</code></pre></td></tr></table>
</div>
</div><p>
        <a data-fancybox="gallery" href="https://gitee.com/stto_32/img/raw/master/20220106145335.png">
            <img class="mx-auto" alt="image-20220106145335208" src="https://gitee.com/stto_32/img/raw/master/20220106145335.png" />
        </a>
    </p>
<p>通过上面的抓包发现Ping的请求包源地址：10.244.123.135，目的地址为：10.244.140.64。 同时发送了两个ARP请求who-has 10.244.123.135 和who-has 169.254.1.1。</p>
<p>同时在<code>node01</code>的eth0（10.10.10.6）网卡上抓包：
tcpdump -i eth0 -ne host 10.244.140.64</p>
<div class="highlight"><div style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4">
<table style="border-spacing:0;padding:0;margin:0;border:0;width:auto;overflow:auto;display:block;"><tr><td style="vertical-align:top;padding:0;margin:0;border:0;">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">1
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">2
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">3
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">4
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">5
</span></code></pre></td>
<td style="vertical-align:top;padding:0;margin:0;border:0;;width:100%">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell"><span style="color:#f92672">[</span>root@WT-SAASMYSQL-10-6 ~<span style="color:#f92672">]</span><span style="color:#75715e">#tcpdump -i eth0 -ne host 10.244.140.64</span>
tcpdump: verbose output suppressed, use -v or -vv <span style="color:#66d9ef">for</span> full protocol decode
listening on eth0, link-type EN10MB <span style="color:#f92672">(</span>Ethernet<span style="color:#f92672">)</span>, capture size <span style="color:#ae81ff">262144</span> bytes
10:05:01.065325 e2:17:1e:22:1a:43 &gt; 9a:40:f3:72:9c:ab, ethertype IPv4 <span style="color:#f92672">(</span>0x0800<span style="color:#f92672">)</span>, length 98: 10.244.123.135 &gt; 10.244.140.64: ICMP echo request, id 7680, seq 0, length <span style="color:#ae81ff">64</span>
10:05:01.066242 9a:40:f3:72:9c:ab &gt; e2:17:1e:22:1a:43, ethertype IPv4 <span style="color:#f92672">(</span>0x0800<span style="color:#f92672">)</span>, length 98: 10.244.140.64 &gt; 10.244.123.135: ICMP echo reply, id 7680, seq 0, length <span style="color:#ae81ff">64</span>
</code></pre></td></tr></table>
</div>
</div><p>上面看到转包的源地址是<code>10.244.123.135（源容器的IP地址）</code>，源MAC是<code>e2:17:1e:22:1a:43</code>，这个Mac地址就是 <code>node01（10.10.10.6）</code>主机的mac地址</p>
<p>上面看到转包的目的地址是<code>10.244.123.135（目的容器的IP地址）</code>，目的MAC是<code>9a:40:f3:72:9c:ab</code>，这个Mac地址就是 <code>node02（10.10.10.108）</code>主机的mac地址</p>
<p>服务器发现要发到10.244.140.64，而<code>10.244.140.64/26 via 10.10.10.108 dev eth0 proto bird</code>指定的网关是<code>10.10.10.108</code>,广播找到<code>10.10.10.108</code>的mac地址是<code>9a:40:f3:72:9c:ab</code>，因此将数据包的目的mac地址改成<code>9a:40:f3:72:9c:ab</code></p>
<p>
        <a data-fancybox="gallery" href="https://gitee.com/stto_32/img/raw/master/20220107101003.png">
            <img class="mx-auto" alt="image-20220107101003418" src="https://gitee.com/stto_32/img/raw/master/20220107101003.png" />
        </a>
    </p>
<p>同时也在node02的eth0(10.10.10.108)网卡上抓包</p>
<div class="highlight"><div style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4">
<table style="border-spacing:0;padding:0;margin:0;border:0;width:auto;overflow:auto;display:block;"><tr><td style="vertical-align:top;padding:0;margin:0;border:0;">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">1
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">2
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">3
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">4
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">5
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">6
</span><span style="margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#7f7f7f">7
</span></code></pre></td>
<td style="vertical-align:top;padding:0;margin:0;border:0;;width:100%">
<pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4"><code class="language-shell" data-lang="shell"><span style="color:#f92672">[</span>root@node02 ~<span style="color:#f92672">]</span><span style="color:#75715e"># tcpdump -i eth0 -ne host 10.244.140.64</span>
tcpdump: verbose output suppressed, use -v or -vv <span style="color:#66d9ef">for</span> full protocol decode
listening on eth0, link-type EN10MB <span style="color:#f92672">(</span>Ethernet<span style="color:#f92672">)</span>, capture size <span style="color:#ae81ff">262144</span> bytes
10:05:13.908051 e2:17:1e:22:1a:43 &gt; 9a:40:f3:72:9c:ab, ethertype IPv4 <span style="color:#f92672">(</span>0x0800<span style="color:#f92672">)</span>, length 98: 10.244.123.135 &gt; 10.244.140.64: ICMP echo request, id 7680, seq 0, length <span style="color:#ae81ff">64</span>
10:05:13.908593 9a:40:f3:72:9c:ab &gt; e2:17:1e:22:1a:43, ethertype IPv4 <span style="color:#f92672">(</span>0x0800<span style="color:#f92672">)</span>, length 98: 10.244.140.64 &gt; 10.244.123.135: ICMP echo reply, id 7680, seq 0, length <span style="color:#ae81ff">64</span>


</code></pre></td></tr></table>
</div>
</div><h3 id="总结两种网络对比">总结：两种网络对比</h3>
<p><strong>IPIP网络</strong>：</p>
<p>流量：tunlo设备封装数据，形成隧道，承载流量。</p>
<p>适用网络类型：适用于互相访问的pod不在同一个网段中，跨网段访问的场景。外层封装的ip能够解决跨网段的路由问题。</p>
<p>效率：流量需要tunl0设备封装，效率略低</p>
<p><strong>BGP网络</strong>：</p>
<p>流量：使用路由信息导向流量</p>
<p>适用网络类型：适用于互相访问的pod在同一个网段，适用于大型网络。</p>
<p>效率：原生hostGW，效率高</p>
<h3 id="存在问题">存在问题 </h3>
<p>(1) 缺点租户隔离问题</p>
<p>Calico 的三层方案是直接在 host 上进行路由寻址，那么对于多租户如果使用同一个 CIDR 网络就面临着地址冲突的问题。</p>
<p>(2) 路由规模问题</p>
<p>通过路由规则可以看出，路由规模和 pod 分布有关，如果 pod离散分布在 host 集群中，势必会产生较多的路由项。</p>
<p>(3) iptables 规则规模问题</p>
<p>1台 Host 上可能虚拟化十几或几十个容器实例，过多的 iptables 规则造成复杂性和不可调试性，同时也存在性能损耗。</p>
<p>(4) 跨子网时的网关路由问题</p>
<p>当对端网络不为二层可达时，需要通过三层路由机时，需要网关支持自定义路由配置，即 pod 的目的地址为本网段的网关地址，再由网关进行跨三层转发。</p>
<h3 id="参考链接">参考链接</h3>
<p><a href="https://cloud.tencent.com/developer/article/1638845">https://cloud.tencent.com/developer/article/1638845</a>
<a href="http://www.asznl.com/post/85">http://www.asznl.com/post/85</a>
<a href="https://cloud.tencent.com/developer/article/1638845">https://cloud.tencent.com/developer/article/1638845</a>
​</p>
        </div>

        
<div class="post-archive">
    <ul class="post-copyright">
        <li><strong>原文作者：</strong><a rel="author" href="https://stto_32.gitee.io/site">仨哥</a></li>
        <li style="word-break:break-all"><strong>原文链接：</strong><a href="https://stto_32.gitee.io/site/post/calico/">https://stto_32.gitee.io/site/post/calico/</a></li>
        <li><strong>版权声明：</strong>本作品采用<a rel="license" href="https://creativecommons.org/licenses/by-nc-nd/4.0/">知识共享署名-非商业性使用-禁止演绎 4.0 国际许可协议</a>进行许可，非商业转载请注明出处（作者，原文链接），商业转载请联系作者获得授权。</li>
    </ul>
</div>
<br/>



        

<div class="post-archive">
    <h2>See Also</h2>
    <ul class="listing">
        
        <li><a href="/site/post/%E5%9F%BA%E4%BA%8E1.22.1%E7%89%88%E6%9C%AC%E7%9A%84k8s%E9%83%A8%E7%BD%B2/">【2022】最详细的k8s基于1.22.1版本部署</a></li>
        
        <li><a href="/site/post/cloudstack&#43;kvm/">cloudstack-4.1.5版本最全入门笔记【2022】</a></li>
        
        <li><a href="/site/post/k8s%E7%8E%AF%E5%A2%83%E4%B8%8Bphp/">k8s环境php环境制品</a></li>
        
        <li><a href="/site/post/kubernetes-client%E4%B9%8Bpython%E8%AF%A6%E7%BB%86%E6%8E%A5%E5%8F%A3%E8%B0%83%E7%94%A8/">kubernetes-client之python详细接口调用</a></li>
        
        <li><a href="/site/post/%E5%9F%BA%E4%BA%8Enfs%E7%9A%84storageclass/">基于nfs的storageclass</a></li>
        
    </ul>
</div>


        <div class="post-meta meta-tags">
            
            <ul class="clearfix">
                
                <li><a href='/site/tags/k8s'>k8s</a></li>
                
                <li><a href='/site/tags/%E4%BA%91%E5%8E%9F%E7%94%9F'>云原生</a></li>
                
            </ul>
            
        </div>
    </article>
    
    <div id="disqus_thread"></div>
<script type="application/javascript">
    var disqus_config = function () {
    
    
    
    };
    (function() {
        if (["localhost", "127.0.0.1"].indexOf(window.location.hostname) != -1) {
            document.getElementById('disqus_thread').innerHTML = 'Disqus comments not available by default when the website is previewed locally.';
            return;
        }
        var d = document, s = d.createElement('script'); s.async = true;
        s.src = '//' + "xull" + '.disqus.com/embed.js';
        s.setAttribute('data-timestamp', +new Date());
        (d.head || d.body).appendChild(s);
    })();
</script>
<noscript>Please enable JavaScript to view the <a href="https://disqus.com/?ref_noscript">comments powered by Disqus.</a></noscript>
<a href="https://disqus.com" class="dsq-brlink">comments powered by <span class="logo-disqus">Disqus</span></a>

    
    
    <div class="post bg-white">
      <script src="https://utteranc.es/client.js"
            repo= "51op/hugoblogtalks"
            issue-term="pathname"
            theme="github-light"
            crossorigin="anonymous"
            async>
      </script>
    </div>
    
</div>

                    <footer id="footer">
    <div>
        &copy; 2022 <a href="https://stto_32.gitee.io/site">Sang的博客 By 仨哥</a>
        
    </div>
    <br />
    <div>
        <div class="github-badge">
            <a href="https://gohugo.io/" target="_black" rel="nofollow"><span class="badge-subject">Powered by</span><span class="badge-value bg-blue">Hugo</span></a>
        </div>
        <div class="github-badge">
            <a href="https://github.com/flysnow-org/maupassant-hugo" target="_black"><span class="badge-subject">Theme</span><span class="badge-value bg-yellowgreen">Maupassant</span></a>
        </div>
    </div>
</footer>


    
    <script type="text/javascript">
        window.MathJax = {
            tex2jax: {
                inlineMath: [['$', '$']],
                processEscapes: true
                }
            };
    </script>
    <script src='https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-MML-AM_CHTML' async></script><script src="https://cdn.bootcdn.net/ajax/libs/fancybox/3.5.7/jquery.fancybox.min.js"></script>

<a id="rocket" href="#top"></a>
<script type="text/javascript" src='/site/js/totop.js?v=0.0.0' async=""></script>



    <script type="text/javascript" src="//busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js" async></script>




    <script src='/site/js/douban.js'></script>

                </div>

                <div id="secondary">
    <section class="widget">
        <form id="search" action='https://stto_32.gitee.io/site/search/' method="get" accept-charset="utf-8" target="_blank" _lpchecked="1">
      
      <input type="text" name="q" maxlength="20" placeholder="Search">
      <input type="hidden" name="sitesearch" value="https://stto_32.gitee.io/site">
      <button type="submit" class="submit icon-search"></button>
</form>
    </section>
    
    <section class="widget">
        <h3 class="widget-title">最近文章</h3>
<ul class="widget-list">
    
    <li>
        <a href="https://stto_32.gitee.io/site/post/calico/" title="Calico网络之BGP详解">Calico网络之BGP详解</a>
    </li>
    
    <li>
        <a href="https://stto_32.gitee.io/site/post/casbin/" title="casbin权限管理">casbin权限管理</a>
    </li>
    
    <li>
        <a href="https://stto_32.gitee.io/site/post/cloudstack&#43;kvm/" title="cloudstack-4.1.5版本最全入门笔记【2022】">cloudstack-4.1.5版本最全入门笔记【2022】</a>
    </li>
    
    <li>
        <a href="https://stto_32.gitee.io/site/post/glusterfs%E5%AE%89%E8%A3%85/" title="glusterfs安装">glusterfs安装</a>
    </li>
    
    <li>
        <a href="https://stto_32.gitee.io/site/post/go%E4%B9%8Bnginx%E6%97%A5%E5%BF%97%E5%88%86%E6%9E%90/" title="go之nginx日志分析">go之nginx日志分析</a>
    </li>
    
    <li>
        <a href="https://stto_32.gitee.io/site/post/%E4%BA%BA%E4%BA%BA%E8%83%BD%E7%9C%8B%E6%87%82%E7%9A%84Async-Go/" title="go异步">go异步</a>
    </li>
    
    <li>
        <a href="https://stto_32.gitee.io/site/post/k8s%E7%8E%AF%E5%A2%83%E4%B8%8Bphp/" title="k8s环境php环境制品">k8s环境php环境制品</a>
    </li>
    
    <li>
        <a href="https://stto_32.gitee.io/site/post/kubernetes-client%E4%B9%8Bpython%E8%AF%A6%E7%BB%86%E6%8E%A5%E5%8F%A3%E8%B0%83%E7%94%A8/" title="kubernetes-client之python详细接口调用">kubernetes-client之python详细接口调用</a>
    </li>
    
    <li>
        <a href="https://stto_32.gitee.io/site/post/mac%E5%B7%A5%E5%85%B7/" title="mac工具">mac工具</a>
    </li>
    
    <li>
        <a href="https://stto_32.gitee.io/site/post/%E5%9F%BA%E4%BA%8E1.22.1%E7%89%88%E6%9C%AC%E7%9A%84k8s%E9%83%A8%E7%BD%B2/" title="【2022】最详细的k8s基于1.22.1版本部署">【2022】最详细的k8s基于1.22.1版本部署</a>
    </li>
    
</ul>
    </section>

    

    <section class="widget">
        <h3 class="widget-title"><a href='/site/categories/'>分类</a></h3>
<ul class="widget-list">
    
</ul>
    </section>

    <section class="widget">
        <h3 class="widget-title"><a href='/site/tags/'>标签</a></h3>
<div class="tagcloud">
    
    <a href="https://stto_32.gitee.io/site/tags/cloudstack/">cloudstack</a>
    
    <a href="https://stto_32.gitee.io/site/tags/k8s/">k8s</a>
    
    <a href="https://stto_32.gitee.io/site/tags/kvm/">kvm</a>
    
    <a href="https://stto_32.gitee.io/site/tags/php/">php</a>
    
    <a href="https://stto_32.gitee.io/site/tags/shell/">shell</a>
    
    <a href="https://stto_32.gitee.io/site/tags/%E4%BA%91%E5%8E%9F%E7%94%9F/">云原生</a>
    
</div>
    </section>

    
<section class="widget">
    <h3 class="widget-title">友情链接</h3>
    <ul class="widget-list">
        
        <li>
            <a target="_blank" href="https://www.cnblogs.com/xull0651/" title="Sang的博客">Sang的博客</a>
        </li>
        
        <li>
            <a target="_blank" href="https://juejin.cn/user/2796746682943495" title="掘金">掘金</a>
        </li>
        
    </ul>
</section>


    <section class="widget">
        <h3 class="widget-title">其它</h3>
        <ul class="widget-list">
            <li><a href="https://stto_32.gitee.io/site/index.xml">文章 RSS</a></li>
        </ul>
    </section>
</div>
            </div>
        </div>
    </div>
</body>

</html>